{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# TensorBoard 举例\n",
    "\n",
    "***\n",
    "**编写** [王何宇](http://person.zju.edu.cn/wangheyu), [浙江大学数学科学学院](http://www.math.zju.edu.cn)\n",
    "\n",
    "**参考资料**\n",
    "1. https://www.tensorflow.org .\n",
    "\n",
    "## Tensorard的启动\n",
    "\n",
    "TensorBoard如果已经配置了路径, 那么直接在终端下输入\n",
    "\n",
    "    tensorboard --logdir=/path/to/log/dir\n",
    "就可以了. `/path/to/log/dir`是在程序代码中指定的. 如果没有配置路径, 那么它的位置在package目录下, 在anaconda中, 是在\n",
    "\n",
    "    anaconda3/pkgs/tensorflow-tensorboard-0.1.5-py36_0/bin\n",
    "下, 其中`tensorflow-tensorboard-`后跟的数字是版本号, 可能不一样. 我们接下去直接把一个基础版本的MNIST项目的代码修改成带流程图和带调试的. 基本上, TensorFlow的项目几乎是必须使用TensorBoard来进行调试的. MNIST的项目代码如下:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "0.9165\n"
     ]
    },
    {
     "ename": "SystemExit",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "An exception has occurred, use %tb to see the full traceback.\n",
      "\u001b[0;31mSystemExit\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/hywang/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py:2870: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.\n",
      "  warn(\"To exit: use 'exit', 'quit', or Ctrl-D.\", stacklevel=1)\n"
     ]
    }
   ],
   "source": [
    "import argparse\n",
    "import sys\n",
    "\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "import tensorflow as tf\n",
    "tf.reset_default_graph()\n",
    "\n",
    "FLAGS = None\n",
    "\n",
    "\n",
    "def main(_):\n",
    "  # Import data\n",
    "  mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)\n",
    "\n",
    "  # Create the model\n",
    "  x = tf.placeholder(tf.float32, [None, 784])\n",
    "  W = tf.Variable(tf.zeros([784, 10]))\n",
    "  b = tf.Variable(tf.zeros([10]))\n",
    "  y = tf.matmul(x, W) + b\n",
    "\n",
    "  # Define loss and optimizer\n",
    "  y_ = tf.placeholder(tf.float32, [None, 10])\n",
    "\n",
    "  # The raw formulation of cross-entropy,\n",
    "  #\n",
    "  #   tf.reduce_mean(-tf.reduce_sum(y_ * tf.log(tf.nn.softmax(y)),\n",
    "  #                                 reduction_indices=[1]))\n",
    "  #\n",
    "  # can be numerically unstable.\n",
    "  #\n",
    "  # So here we use tf.nn.softmax_cross_entropy_with_logits on the raw\n",
    "  # outputs of 'y', and then average across the batch.\n",
    "  cross_entropy = tf.reduce_mean(\n",
    "      tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))\n",
    "  train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)\n",
    "\n",
    "  sess = tf.InteractiveSession()\n",
    "  tf.global_variables_initializer().run()\n",
    "    \n",
    "  # Train\n",
    "  for _ in range(1000):\n",
    "    batch_xs, batch_ys = mnist.train.next_batch(100)\n",
    "    sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})\n",
    "\n",
    "  # Test trained model\n",
    "  correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))\n",
    "  accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "  print(sess.run(accuracy, feed_dict={x: mnist.test.images,\n",
    "                                      y_: mnist.test.labels}))\n",
    "  # 这里输出了流程图.\n",
    "  writer = tf.summary.FileWriter(\"tensorboard/mnist_simple/0\", graph=tf.get_default_graph())\n",
    "  writer.close()\n",
    "\n",
    "if __name__ == '__main__':\n",
    "  parser = argparse.ArgumentParser()\n",
    "  parser.add_argument('--data_dir', type=str, default='MNIST_data',\n",
    "                      help='Directory for storing input data')\n",
    "  FLAGS, unparsed = parser.parse_known_args()\n",
    "tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "上面这段代码和官方Github中唯一不一样的就是增加了\n",
    "\n",
    "    writer = tf.summary.FileWriter(\"tensorboard/mnist_simple/0\", graph=tf.get_default_graph())\n",
    "    writer.close()\n",
    "第一句指定在当前路径的`tensorboard/mnist_simple/0`下输出流程图. 第二句关闭书写器. 运行后, 在指定的目录中会有一个名为\n",
    "\n",
    "    events.out.tfevents.1515833649.macaque\n",
    "的`events`文件, 这里具体的数字会不同. 如果这个文件没有出现, 要检查一下write的地方是否正确指定了目录. 我们现在就可以看流程图了:\n",
    "\n",
    "    tensorboard --logdir=tensorboard/mnist_simple/0\n",
    "应该是这样的结果.\n",
    "![](images/mnist_simple_0.png)\n",
    "很混乱啊. 所以我们需要整理我们的代码, 并给它加标记. 这不仅仅是为了可视化, 合理的标记也有助于我们整理运算逻辑. 比如我们注意到代码中并没有在空间上区分placeholder和Variable, 而是混在了一起. 这个既不利于可视化, 也不利于逻辑推导. 我们稍微整理一下, 然后各自加个标记."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "0.9211\n"
     ]
    },
    {
     "ename": "SystemExit",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "An exception has occurred, use %tb to see the full traceback.\n",
      "\u001b[0;31mSystemExit\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/hywang/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py:2870: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.\n",
      "  warn(\"To exit: use 'exit', 'quit', or Ctrl-D.\", stacklevel=1)\n"
     ]
    }
   ],
   "source": [
    "import argparse\n",
    "import sys\n",
    "\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "import tensorflow as tf\n",
    "tf.reset_default_graph()\n",
    "\n",
    "FLAGS = None\n",
    "\n",
    "def main(_):\n",
    "    mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)\n",
    "\n",
    "    # Import data\n",
    "    with tf.name_scope('input'):\n",
    "        x = tf.placeholder(tf.float32, [None, 784])\n",
    "        y_ = tf.placeholder(tf.float32, [None, 10])\n",
    "\n",
    "\n",
    "    with tf.name_scope('model'):\n",
    "        W = tf.Variable(tf.zeros([784, 10]))\n",
    "        b = tf.Variable(tf.zeros([10]))\n",
    "        y = tf.matmul(x, W) + b\n",
    "\n",
    "    cross_entropy = tf.reduce_mean(\n",
    "    tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))\n",
    "    train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)\n",
    "\n",
    "    sess = tf.InteractiveSession()\n",
    "    tf.global_variables_initializer().run()\n",
    "    \n",
    "    # Train\n",
    "    for _ in range(1000):\n",
    "        batch_xs, batch_ys = mnist.train.next_batch(100)\n",
    "        sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})\n",
    "\n",
    "    # Test trained model\n",
    "    correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))\n",
    "    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "    print(sess.run(accuracy, feed_dict={x: mnist.test.images,\n",
    "                                      y_: mnist.test.labels}))\n",
    "    # 这里输出了流程图.\n",
    "    writer = tf.summary.FileWriter(\"tensorboard/mnist_simple/1\", graph=tf.get_default_graph())\n",
    "    writer.close()\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    parser = argparse.ArgumentParser()\n",
    "    parser.add_argument('--data_dir', type=str, default='MNIST_data',\n",
    "                        help='Directory for storing input data')\n",
    "    FLAGS, unparsed = parser.parse_known_args()\n",
    "\n",
    "tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们观察一下新的流程图, \n",
    "\n",
    "    tensorboard --logdir=tensorboard/mnist_simple/1\n",
    "![](images/mnist_simple_1.png)\n",
    "我们注意到右边确实多了两个模块, 一个叫input, 一个叫model, 点开还能看细节. 不过这些细节的名称也很混乱. 比如placehold和variable都是用placeholder1, placeholder2这样的名字. 这是应为在代码中我们没有给它们取输出的名字. TensorFlow中所有对象都可以给一个输出命名. 我们来改一下."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "0.9168\n"
     ]
    },
    {
     "ename": "SystemExit",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "An exception has occurred, use %tb to see the full traceback.\n",
      "\u001b[0;31mSystemExit\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/hywang/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py:2870: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.\n",
      "  warn(\"To exit: use 'exit', 'quit', or Ctrl-D.\", stacklevel=1)\n"
     ]
    }
   ],
   "source": [
    "import argparse\n",
    "import sys\n",
    "\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "import tensorflow as tf\n",
    "tf.reset_default_graph()\n",
    "\n",
    "FLAGS = None\n",
    "\n",
    "def main(_):\n",
    "    mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)\n",
    "\n",
    "    # Import data\n",
    "    with tf.name_scope('input'):\n",
    "        x = tf.placeholder(tf.float32, [None, 784], name=\"picture\")\n",
    "        y_ = tf.placeholder(tf.float32, [None, 10], name=\"label\")\n",
    "\n",
    "\n",
    "    with tf.name_scope('model'):\n",
    "        W = tf.Variable(tf.zeros([784, 10]), name=\"weight\")\n",
    "        b = tf.Variable(tf.zeros([10]), name=\"bias\")\n",
    "        y = tf.matmul(x, W) + b\n",
    "\n",
    "    cross_entropy = tf.reduce_mean(\n",
    "    tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))\n",
    "    train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)\n",
    "\n",
    "    sess = tf.InteractiveSession()\n",
    "    tf.global_variables_initializer().run()\n",
    "    \n",
    "    # Train\n",
    "    for _ in range(1000):\n",
    "        batch_xs, batch_ys = mnist.train.next_batch(100)\n",
    "        sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})\n",
    "\n",
    "    # Test trained model\n",
    "    correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))\n",
    "    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "    print(sess.run(accuracy, feed_dict={x: mnist.test.images,\n",
    "                                            y_: mnist.test.labels}))\n",
    "\n",
    "    writer = tf.summary.FileWriter(\"tensorboard/mnist_simple/2\", graph=tf.get_default_graph())\n",
    "    writer.close()\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    parser = argparse.ArgumentParser()\n",
    "    parser.add_argument('--data_dir', type=str, default='MNIST_data',\n",
    "                        help='Directory for storing input data')\n",
    "    FLAGS, unparsed = parser.parse_known_args()\n",
    "\n",
    "tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们观察一下新的流程图, \n",
    "\n",
    "    tensorboard --logdir=tensorboard/mnist_simple/2\n",
    "![](images/mnist_simple_2.png)\n",
    "现在细节也对了. 我们按这个逻辑, 把整个程序整理一下."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "0.9183\n"
     ]
    },
    {
     "ename": "SystemExit",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "An exception has occurred, use %tb to see the full traceback.\n",
      "\u001b[0;31mSystemExit\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/hywang/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py:2870: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.\n",
      "  warn(\"To exit: use 'exit', 'quit', or Ctrl-D.\", stacklevel=1)\n"
     ]
    }
   ],
   "source": [
    "import argparse\n",
    "import sys\n",
    "\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "import tensorflow as tf\n",
    "tf.reset_default_graph()\n",
    "\n",
    "FLAGS = None\n",
    "\n",
    "def main(_):\n",
    "    mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)\n",
    "\n",
    "    # Import data\n",
    "    with tf.name_scope('input'):\n",
    "        x = tf.placeholder(tf.float32, [None, 784], name=\"picture\")\n",
    "        y_ = tf.placeholder(tf.float32, [None, 10], name=\"label\")\n",
    "\n",
    "\n",
    "    with tf.name_scope('model'):\n",
    "        W = tf.Variable(tf.zeros([784, 10]), name=\"weight\")\n",
    "        b = tf.Variable(tf.zeros([10]), name=\"bias\")\n",
    "        y = tf.matmul(x, W) + b\n",
    "\n",
    "    with tf.name_scope('loss'):\n",
    "        cross_entropy = tf.reduce_mean(\n",
    "        tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))\n",
    "        train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)\n",
    "\n",
    "    sess = tf.InteractiveSession()\n",
    "    tf.global_variables_initializer().run()\n",
    "    \n",
    "    with tf.name_scope('train'):\n",
    "        # Train\n",
    "        for _ in range(1000):\n",
    "            batch_xs, batch_ys = mnist.train.next_batch(100)\n",
    "            sess.run(train_step, feed_dict={x: batch_xs, y_: batch_ys})\n",
    "\n",
    "    with tf.name_scope('test'):\n",
    "        # Test trained model\n",
    "        correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))\n",
    "        accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "        print(sess.run(accuracy, feed_dict={x: mnist.test.images,\n",
    "                                            y_: mnist.test.labels}))\n",
    "\n",
    "    writer = tf.summary.FileWriter(\"tensorboard/mnist_simple/3\", graph=tf.get_default_graph())\n",
    "    writer.close()\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    parser = argparse.ArgumentParser()\n",
    "    parser.add_argument('--data_dir', type=str, default='MNIST_data',\n",
    "                        help='Directory for storing input data')\n",
    "    FLAGS, unparsed = parser.parse_known_args()\n",
    "\n",
    "tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "现在我们观察一下新的流程图, \n",
    "\n",
    "    tensorboard --logdir=tensorboard/mnist_simple/3\n",
    "![](images/mnist_simple_3.png)\n",
    "非常整洁了, 而且我们能清晰地看到数据是如何在各模块间流动的."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Extracting MNIST_data/train-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/train-labels-idx1-ubyte.gz\n",
      "Extracting MNIST_data/t10k-images-idx3-ubyte.gz\n",
      "Extracting MNIST_data/t10k-labels-idx1-ubyte.gz\n",
      "0.9198\n"
     ]
    },
    {
     "ename": "SystemExit",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "An exception has occurred, use %tb to see the full traceback.\n",
      "\u001b[0;31mSystemExit\u001b[0m\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/home/hywang/anaconda3/lib/python3.6/site-packages/IPython/core/interactiveshell.py:2870: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.\n",
      "  warn(\"To exit: use 'exit', 'quit', or Ctrl-D.\", stacklevel=1)\n"
     ]
    }
   ],
   "source": [
    "import argparse\n",
    "import sys\n",
    "\n",
    "from tensorflow.examples.tutorials.mnist import input_data\n",
    "\n",
    "import tensorflow as tf\n",
    "tf.reset_default_graph()\n",
    "\n",
    "FLAGS = None\n",
    "\n",
    "def main(_):\n",
    "    mnist = input_data.read_data_sets(FLAGS.data_dir, one_hot=True)\n",
    "\n",
    "    # Import data\n",
    "    with tf.name_scope('input'):\n",
    "        x = tf.placeholder(tf.float32, [None, 784], name=\"picture\")\n",
    "        y_ = tf.placeholder(tf.float32, [None, 10], name=\"label\")\n",
    "        \n",
    "    with tf.name_scope('input_reshape'):\n",
    "        # 这里要把图像转回28x28x1的灰度格式.\n",
    "        image_shaped_input = tf.reshape(x, [-1, 28, 28, 1])\n",
    "        tf.summary.image('input', image_shaped_input, 20)\n",
    "\n",
    "    with tf.name_scope('model'):\n",
    "        W = tf.Variable(tf.zeros([784, 10]), name=\"weight\")\n",
    "        b = tf.Variable(tf.zeros([10]), name=\"bias\")\n",
    "        y = tf.matmul(x, W) + b\n",
    "    with tf.name_scope('bias'):     \n",
    "        tf.summary.scalar('max_b', tf.reduce_max(b))\n",
    "        tf.summary.scalar('min_b', tf.reduce_min(b))\n",
    "        tf.summary.histogram('histogram', b)\n",
    "        \n",
    "    with tf.name_scope('watch_weight'):\n",
    "        watch_weight = tf.reshape(tf.transpose(W), [10, 28, 28, 1])\n",
    "        tf.summary.image('weight', watch_weight, 10)\n",
    "\n",
    "    with tf.name_scope('loss'):\n",
    "        cross_entropy = tf.reduce_mean(\n",
    "        tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y))\n",
    "        train_step = tf.train.GradientDescentOptimizer(0.5).minimize(cross_entropy)\n",
    "        tf.summary.scalar('cross_entropy', cross_entropy)\n",
    "\n",
    "    sess = tf.InteractiveSession()\n",
    "    tf.global_variables_initializer().run()\n",
    "\n",
    "    # 收集一下全部的summary\n",
    "    merged = tf.summary.merge_all()\n",
    "    # 写流程图\n",
    "    writer = tf.summary.FileWriter(\"tensorboard/mnist_simple/4\", sess.graph)\n",
    "\n",
    "    with tf.name_scope('train'):\n",
    "        # Train, 这是最需要调试的过程\n",
    "        for i in range(1000):\n",
    "            batch_xs, batch_ys = mnist.train.next_batch(100)\n",
    "            summary, _ = sess.run([merged,train_step], feed_dict={x: batch_xs, y_: batch_ys})\n",
    "            #summary = sess.run(merged, feed_dict={x: batch_xs, y_: batch_ys})\n",
    "            writer.add_summary(summary, i)\n",
    "\n",
    "    with tf.name_scope('test'):\n",
    "        # Test trained model\n",
    "        correct_prediction = tf.equal(tf.argmax(y, 1), tf.argmax(y_, 1))\n",
    "        accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))\n",
    "        print(sess.run(accuracy, feed_dict={x: mnist.test.images,\n",
    "                                            y_: mnist.test.labels}))\n",
    "\n",
    "    writer.close()\n",
    "\n",
    "if __name__ == '__main__':\n",
    "    parser = argparse.ArgumentParser()\n",
    "    parser.add_argument('--data_dir', type=str, default='MNIST_data',\n",
    "                        help='Directory for storing input data')\n",
    "    FLAGS, unparsed = parser.parse_known_args()\n",
    "\n",
    "tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.1"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
